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October 23, 1981 


The Hon. Harry Fitzwater 

Deputy Director for Administration 
Central Intelligence Agency 
Washington, D.C. 20505 


Dear Mr. Fitzwater: 


ae I are pleased to submit the attached report, 
"Psychologists in the Central Intelligence Agency." We are indebted to the men 


and women of the Agency who cooperated in this study by furnishing written material 
and sharing with the study team their observations and judgments. Throughout the 
course of the study we received the enthusiastic assistance of those contacted. 


Two issues dealt with in this report deserve a further word of elaboration: 
(1) how to assure "quality control" of psychological activities, and (2) how to 
improve communications among psychologists. In the course of our wrap-up discussions, 
concern was raised about the lack of a central point for the "professional super- 
vision" of psychological activities to avoid needless embarrassment of activities 
that are poorly planned or conducted or that are ill-advised in the first place. 
This need for quality control can be approached in two quite different ways. One 
is to require that all activities be scrutinized and approved at a single point, on 
behalf of the Agency--presumably by a psychologist. The second is to institute 
systematic means for evaluating psychological activities within the context of the 
organization whose mission is being supported. In the latter case judgment is 
rendered by the manager of the unit being supported and related to its mission as 
it fits into Agency priorities. We believe this approach to quality control is 
more reliable and effective than having a "gate-keeper" whose judgment is likely to 


be influenced by his or her own professional biases. 


The question of how best to improve communications among psychologists 
is particularly challenging because of the wide variety of interests and responsi-~ 
bilities, and few incentives to stimulate such interchange. Our approach is to 
identify one or more central purposes which reflect common interests or problems 
toward which each group might contribute ina positive fashion. We believe that an 
enforced requirement for systematic program evaluation could provide such a central, 
common purpose. Another might be the sponsoring of symposia from time to time on 
new developments or problems of particular interest in the Agency in the field of 
psychology. One possibility, not addressed in this report, is the potential for 
improved interchange that could accompany the establishment of a "career service" 
for psychologists. <A career service might offer improved opportunity for inter- 
change via some job rotation; tut this should be tempered by the mumber of 
one-person operations. The question on the desirability of establishing a career 
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The Hon. Harry Fitzwater -2~ October 23, 1981 


service for psychologists requires a different approach from this study which 
addressed the utilization of psychologists from the perspective of organizational 
location. 


[|__| I were impressed with the quality and commitment of 
the professional staff in the Agency. They certainly are a credit to the public 25X4 
service of the United States. 


Sincerel 
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EXECUTIVE SUMMARY 


The general objective of this project was to study all of the 
Agency's psychological efforts and to evaluate how effectively and 
efficiently they are meeting the Agency's needs’ in the psychological 
field. 


This study is directed at the 39 individuals who are working 
specifically as psychologists regardless of job title (such as 
''behavioral scientist,'' etc.). It does not include those trained as 
psychologists but not presently working in the field of psychology. 
Psychologists in the Agency perform one or more of five principal 
functions: assessment, research, human factors analysis, training, and 


counseling. 


During the course of the study, 45 individuals were 
interviewed--psychologists (as defined above), their supervisors, and 


some of the clients served by psychologists. 


Conclusions and Recommendations 

1. The work of psychologists in the Agency is sharply defined 
and follows closely the missions and functions of those organizations 
in which they are located. There is no significant overlap that would 
contribute to poor utilization on the basis of organizational 


location. 
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Recommendation: Consolidation of psychologists within the Agency 


would serve no useful purpose, and could disrupt the effective conduct 


of present functions. There may be instances where some 
organizational adjustment would improve the utilization of 
psychologists. 


2. Relative effectiveness in the utilization of psychologists 
was difficult to assess because there is no common practice from one 
organization to another for collecting data that would support such an 
assessment. Information about effectiveness varies greatly and tends 
to be aneeaaesD and unsystematic. Psychologists appear to be busy and 
‘productively employed. Where caseload data is applicable, such as in 
the assessment function, the number of cases has increased by 25 to 50 
percent in the last few years as Agency recruitment efforts have 
intensified and as client organizations have requested assessments on 
a larger proportion of candidates. The study team believes that 
psychologists are not underutilized. However, we are notin a 
position to judge whether, in the light of Agency-wide or Directorate- 
wide priorities, they are at work on the most important tasks. 


Recommendation: Agency management should be able to determine if 


the distribution of effort in these psychological units coincides with 
Agency priorities. We find no particular reason to focus’ upon 
psychologists; determining proper’ distribution of effort is a 
management responsibility applicable to all of the Agency's human 


resources. 
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3. Systematic program evaluation is uneven among the 
psychological units. As one might anticipate, the incentive for 
program evaluation is stronger among those units most heavily involved 
in testing and assessment than in those concentrating more upon 
research, Clinical evaluation and counseling, and training. All units 
engage in some qualitative evaluation, such as user or client 
But teenceionset ses does the client use the product, or does the client 
return for future work? In at least one instance |] more 
systematic and methodologically relevant evaluation is hindered by 
security considerations. In this instance, we believe that ways can 
be found to make needed data available without revealing sensitive 
sources, but only if senior managers recognize this need and provide 
the organizational leverage to carry it out. 

Recommendation: Agency management should require each 
psychological unit to develop and to conduct a systematic program of 
evaluation. 

4. Systematic internal research to determine those aspects of a 
unit's activities most related to success was notably lacking. Where 
there was program evaluation, it rarely was carried the next step to 
systematically identify what elements in the particular task or 
objective contributed most to success. 

Recommendation: Agency management should encourage psychological 
units (and others) to undertake internal research. Such encouragement 


could be pursued through the budget, program review, and performance 
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assessment processes. 

5. Interchange among payenaledietes across organizational lines, 
has been encouraged by Agency management from time to time, but with 
few results. The process suffers from inadequate incentives and the 
press of immediate responsibilities or diverging interests. The study 
team found a rich variety of activities that should be of interest 
among psychologists throughout the Agency. what is needed is a 
problem or issue to which all can make a contribution, and which has 
elements of common interest. 

Recommendation: Tf Agency management makes a serious attempt to 
require (or encourage) systematic evaluation and internal research, 
psychologists, wherever located, will have abundant opportunity to 
seek mutual support and ideas in order to address the challenges of 
methodology and technique associated with successful evaluation and 


related research. 
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I. INTRODUCTION 


The general objective of this project was to study all of the 
“Agency's psychological efforts and to evaluate how effectively and 
efficiently they are meeting the Agency's needs in the psychological 


field. 


25X1 
This project was conducted, under contract with the Agency, by 


background is in public administration; is in psychology. 


This study was undertaken largely because of the recommendation 
by the Inspector General following recent periodic reviews of ene ay 


25X11 ; ; : 
Office of Medical Services. 


There was a brief period in the Agency's history when both of 


these groups were located in a single organization, the Assessment and 
Evaluation Staff of the Office of Training (1951-1954). Prior to 1951 
the A&E staff was located in the Directorate of Plans, the precursor 
of the Operations Directorate. It provided testing and psychological 


assessment support, mainly for recruiting and staffing, but also for 
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In 1951 the Office of Training (OTR) was established as a free 
standing organization to service the entire agency, and the A&E staff 
was moved to OTR, but remained within a group devoted to support of 
DD/O. In 1953 OTR was reorganized and the A&E staff no longer was 


25X1 more or less exclusively dedicated to DD/O needs, but to Agency-wide 


requirements. 


Between 1951 and 1962 there were a series of organizational 
tempests between the A&E staff and the Psychiatric Division in the 
Office of Medical Services. Following a number of key staff changes, 
the A&E staff was transferred, as an entity, to OMS--a transfer 
resulting from a recommendation in the 1959 report of the Inspector 


General on the A&E staff. 
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In conducting this project the study team collected data from 
three sources: historical records and reports (to the extent that 
they were available, including mission statements and position 
descriptions); personnel and staffing data provided by the Office of 
Personnel Policy, Planning, and Management; and, personal interviews 
with psychologists, their supervisors and clients. The interviews 
were directed at eliciting information and views about the nature of 
work actually undertaken by psychologists, the relationship of such 
work to the parent organization's mission, and how such work was 
useful or supportive to those using it. The study team interviewed 45 
professional employees throughout the Agency during the course of the 
study (see Appendix A for the list of those contacted). Every effort 
was made by the study team to cross check information and views 


collected to assure validity. 
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II. CURRENT ORGANIZATIONAL POSITIONS OF PSYCHOLOGISTS 


Before describing where psychologists are located throughout the 
Agency, it is necessary to determine who are psychologists. The study 
team found four different perspectives in defining exactly what 
persons ought to be the focal group for the study. First, there are 
those professionals who currently are performing tasks as 
psychologists; that is, they are in jobs where the skills of a 
psychologist are particularly useful and may be a formal requirement 
for legitimately holding that position. There are 39 professionals 


that can be designated as psychologists by this criterion. 


Second, there are those persons employed by the Agency in 
professional positions, where the incumbent holds a graduate degree 
(Masters or PhD) in psychology. There are 62 professionals throughout 


the Agency so identified by OPPPM records. 


Third, there are those employees who are classified within the 
Agency personnel system as psychologists. These positions carry an 
occupational identity in the 180 series of personnel classification 
and carry the title: psychologist-general, psychologist-assessment, 
psychologist-research, psychologist-clinical, or behavioral scientist. 


The last category is not exclusive to psychologists since it includes 


sociologists, anthoropologists, and political scientists. However, 
the study team, with the assistance of supervisors, screened this 
8 
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category so that our count includes only those behavioral scientists 


who are psychologists. There are 39 psychologists in the 180 series. 


Finally, there are those persons occupying positions carrying the 
-occupational title of psychologist--this group closely parallels the 
third group, but the data run identified only 33 occupying such 


positions. 


Because this study is directed at those who are working as 
psychologists, that definition was used by the study team to collect 
information regarding tasks, missions, value and use of the work 
undertaken. In this process some attention was given to those 
specific instances where psychologists made substantial utilization of 
their skills, but were not so identified, and where a professional 
performed as a psychologist during his or her Agency career but who 
has recently moved into a management or related position where 
psychology is no longer required for the job. 

Professionals, working as psychologists, are located in 
organizations of each of three directorates: Administration (21), 

ae Po and National Foreign Assessment (4). 
(Note: see Appendix B for the full list of individuals by organization 
and classification). These figures reflect only filled positions as 
of the end of September, and do not include persons who work less than 


full time or who, though employed extensively, are consultants. 


All of the psychologists within the DD/A are located in the 
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Office of Medical Services. By organization within OMS they break out 
as follows: Psychiatric Division (3), Psychological Services Division 


: Sears 25X1 
(17), and the Center for Counterterrorism and Crisis Response (1). 


Although not included in the formal definition used here, nor in 
the count of professionals working as psychologists, special note must 
be made of six psychologists in the Office of Training and Education 
(OTE). All are in the Management Training Branch and hold graduate 


degrees in psychology. Their occupational titles are: Training 
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Officer, Instructor-Administration, and Operation Research Analyst. 
All are engaged in developing, conducting or supervising management 
training--an area of endeavor that typically attracts many 
psychologists and which is greatly enriched by their research and 
professional skills. The group does its own training Aviat i Bing a 
function that once was done by PSD's predecessor, the Assessment and 


Evaluation Staff, nearly twenty years ago. 
25X1 


Several employees of considerable tenure came to work at’ the 
Agency as psychologists and performed in that capacity before moving 


For instance 


In summary, psychologists--those working as psychologists (who 


also hold degrees in psychology)--are employed in eight different 


organizations at the branch or division level within the three 
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25X11 
directorates of: Administration and 
National Foreign Assessment In addition there are six 
psychologists in the Management Training Branch devoted exclusively to 
that work. en 
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III. FUNCTIONS PERFORMED BY PSYCHOLOGISTS 


The activities of psychologists in this Agency fall into five basic 


categories of functions: 


A. Assessment 

B. Research 

C. Human factors analyses 
D. Training 

E. Counseling 


The tasks performed under each of these functions are presented below. 
The location within the Agency where these tasks are performed is referred 
to as a'psychological unit." The term "psychological unit" is being applied 
to any place in the Agency in which psychologists are working professionally 
as psychologists. (There are some individuals with advanced degrees in 
psychology who do not work as psychologists. There are some psychologists 
who work as a psychologist but have a job title other than "'psychologist"-- 
an example is "behavioral scientist.) This definition of a psychological unit 
includes places where the entire professional staff is made up of psychologists 
and those where only one psychologist is on a staff composed of other 


professional disciplines. 
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The foregoing presents the many functions performed by psychologists 
in the Agency and the units in which the respective functions are 
performed. One can gain organizational perspective by reorganizing this 
information according to the units where the functions are performed. 
Psychiatric Division 
Office of Medical Services 


Directorate of Administration 


1. Psychiatric assessment of all 
applicants to Agency ; 


2. Clinical assessment of Agency 
personnel with psychiatric problems 


3. Program of research on stress 


4. Development of clinical tests 
for psychiatric evaluations of 
applicants 


5. Psychiatric counseling for 
Agency personnel 


Psychological Services Division 
Office of Medical Services 
Directorate of Administration 


1. Assessment of professional applicants 
to Agency 


2. Assessment of non-professional personnel 
for upward mobility 


3. Assessment activities associated 
with programs of Office of Training 
and Education 


4. Individual intensive personality 
assessment related to career training 
and special assignments 


5. Assessment of photointerpretation 
aptitude 


6. Vocational and career counseling, 
pre-retirement counseling 


7. Assessment of individual progress on the 
Creative Management Program conducted by 
the Office of Training and Education 


Toss 
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8. Psychometric research (validity, 
reliability, etc.) 


9. Research on professional applicant 
procedures that differentiate 
between high versus low performers 


10. Research on effectiveness of 
career counseling procedures 


11. Survey research to determine 
perceptions of and reactions to 
intelligence 


12. Skills analysis of National Foreign 
Assessment Center personnel to 
determine capabilities within that 
unit 
13. 25X11 


25X11 


Center for Counterterrorism and Crisis Response 
Office of Medical Services 
Directorate of Administration 
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IV ASSESSMENT OF THE ROLE AND UTILIZATION OF PSYCHOLOGISTS 


A. What are the Similarities and Differences of Missions in 


Organizations Employing Psychologists? 


Each organization that employs psychologists has its explicit 
mission. The work performed by psychologists is governed by the 
missions of the units in which they are located. It is only in an 
over-generalized sense that it can be said that there are similarities 
among missions. This is demonstrated by reviewing the respective 
missions of the Psychiatric Division of the Office of Medical 


Services, the Psychological Services Division of the same Office and 


The Psychiatric Division appraises all applicants within the 
context of medical qualification. The final decision about an 


applicant, who undergoes assessment by the Office of Medical Services, 


is a medical judgment that includes’ physical and psychiatric 
assessment. 
Although also in the Office of Medical Services, the 
21 
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Psychological Services Division has the mission of providing across-— 
the-board applied psychological services to those units in the Agency 
that desire its services, One of these services is to provide 
assessment information on applicants to professional positions. This 
information is sent to prospective employers and they make the final 
decisions as to hiring applicants. The Division's mission also 
includes providing applied psychological research services to th 


©O5xX1 
Agency. 
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Psychologists in the Research Branch of the Psychological 
Services Division provide support service to units within the Agency 
that have human factor problems associated with present systems or 


those about to be installed, e.g. SAFE. 


Each organization within the Agency has its explicitly defined 
mission. The work of each organization's professionals, not just 
psychologists, is focused upon its particular mission. As might be 
expected, the study team found that, in general, psychologists were 
most effective when they concentrated their attention upon the 


specific functions that are defined by their organization's mission. 
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B. How are the Functions Performed by Psychologists Similar or 
Different Among Organizations Employing Them? 


1. Assessment 


Four groups within the Agency perform assessments, using a 
variety of instruments and techniques, aimed at describing or 
analyzing human behavior. Each is directed at different populations 
or undertaken for different purposes. The Psychiatric Division (PD) 
conducts tests and analyses to identify and diagnose pathological or 
potential pathological behavior in candidates seeking employment with 25x1 
rior to an 


the Agency, and among e r_their families referred 


overseas assignment. 


The Psychological Services Division (PSD) screens, through 
testing and assessment, about 70 percent of professional candidates 
for employment by the Agency. This is directed at skills, knowledges, 
and abilities for a vocational profile to identify those with the best 
fit for particular jobs, for upward mobility programs, and _ fo 


3 
25X1 
retirement counseling. It is vocationally-oriented, not clinical. 
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4. Training 
Three organizations undertake significant training activities, 
25X11 | pre Office of Training and Education (OTE) and the Center for 
Counterterrorism and Crisis Response (CCCR). OTE is the focal point 
for Agency-wide training and education programs. PSD collaborates 
with OTE in courses on creative management and supervisory SAMBO INS ye 
25x! | __ pre element in the Career Trainee Program on human elements in 
operational behavior. 
25X1 
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organizations are substantially different. The common thread is that 


it is predominantly based upon knowledge of human behavior. 
5. Counseling 


Counseling tends to be limited to PD and PSD, although CCCR does 
counsel or advise Agency and other U.S. government client agencies on 
counterterrorism crisis involving hostages, etc. The line between the 
kind of counseling in PD and PSD is quite sharply drawn. As a rule, 
PD concentrates upon clinical psychiatric counseling and treatment. 


PSD focuses upon vocational, career and pre-retirement counseling 


related to selection for hire, career development, upward mobility, 

and retirement. Admittedly there is, from time to time, some ad hoc 

personal counseling in both PSD and but it is not frequent and 25x1 

clearly is discouraged by PSD management. 5X4 
28 
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C. Organizational and Management Issues 


The first issue is at the heart of this study: Should 
psychologists in the Agency be consolidated into a Single 
organization? The history of PSD through 1966 (The Psychological 


Services Staff of the Office of Medical Services, 1951-1966, 


[august 1973) and the reports of Inspector General on 


25X1 


(1980) and OMS (1981) suggest several possible rationales for such 45x14 


move. Consolidation presumably would: (1) avoid overlap and 
duplication of functions and resources, (2) permit better utilization 
of specially trained personnel, and (3) have the potential to improve 
career opportunities for psychologists. The study team believes that 
none of these benefits would accrue from consolidation, and further, 
that consoldation probably would create more problems than it was 


designed to solve. 


Our review reveals that overlap and duplication of functions is 
more apparent than real. Although many elements of the five functions 
performed by psychologists are generic in nature, the specific 
applications, the particular methods used, and the environment in 
which they operate--all contribute to different organizational 


missions that could prove unmanageable for a single organization. 


Second, we found no glaring examples of ‘''underutilization.'! 
This is to be distinguished from possible misapplication, which is 
more likely in a large group than in a smaller one, if only because it 


can more easily be overlooked. 
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Third, the dispersion of psychologists actually favors career 
opportunities. Advancement in CIA beyond GS-14 usually requires the 
person to demonstrate managerial or executive talent beyond his or her 
particular specialty. Putting all psychologists into a single 
organization would provide less opportunity for broad contacts 
throughout the Agency, reduce an individual's visibility, and 
encourage retreat within that specialty or movement outside the 
Agency. It is true that the performance of psychologists now often is 
judged by non-psychologists--but then most of the clients are not 
professional psychologists. In short, our review reveals no good 


reason for consolidating psychologists in the Agency. 


Another concern of this study was the effective utilization of 
professional psychologists. This is much more difficult to answer 
simply, in part because there is no common practice from one 
organization to another for collecting hard data that would permit a 
valid basis for program evaluation. That evaluation which is done 
tends to be anecdotal and unsystematic. Or it relies on gross data, 
“such as cases handled, without accompanying qualitative information 
that would erovias real meaning. As noted above, psychologists 
throughout the Agency appear to be busy. The key question is: Are 
they busy on activities of highest priority to the Agency? On this we 


found some positive clues, but more questions than answers. 


For example, two different approaches are taken with regard to 


research priorities in the two organizations conducting or managing 
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most psychologically oriented research--ORD and PSD. ORD annually 
sol velte its clients for their ideas on what they consider the most 
important problems for research. To this is added ORD suggestions on 
what technology appears ripe for exploitation in view of client 
interests. Through a process of discussion and negotiation a research 
agenda is hammered out. The Research Branch of PSD is obliged to meet 
the research requirements of PSD first. Then it can respond to 
requests for assistance’ from other Agency organizations. PSD 
currently provides research assistance to various organizations in 
DD/NFA, to PD, to the Office of Deveopment and Engineering, and to 
NPIC, among others. These requests are handled as they arrive with 
the judgment being made on whether sufficient resources are available, 
in the light of other commitments, to undertake the work. The ORD 
system is more systematic in a formal sense, but this does not mean 
that PSD's more informal determination is less effective in applying 


the resources of its small (six person) staff. 
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A third concern has been whether or not there are commom 
interests or problems that psychologists might address, regardless of 
organizational lines. At the moment there is little continuing 
collaboration among psychologists located in different organizations. 
From time to time individual psychologists discover a colleague 
elsewhere with similar interests, and a productive dialogue ensues. 


The study team found a great deal of exciting work under way from 


which a productive interchange could be encouraged. Stimulation and 
encouragement from senior officials of the Agency are needed. Also 
needed is some common focus to evoke collaboration--not more 
management directives or reporting requirements. Part D, below, 


suggests such a topic, as one vehicle to stimulate greater 


interchange. 
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D. Need for Evaluation and Internal Research 


era 


Each of the respective psychological units has critical 
responsibilities. Each provides services that have direct, and 
indirect, impact throughout the Agency in important ways. One might 
expect that the top administration would require each psychological 
unit to develop and maintain accountability systems and methods of 
organizational quality control. Such a mandate does not seem to 
exist. The psychological units are, in varying degrees, sensitive to 
these particular issues, but they differ with respect to the effort 


they put into these matters. 


tesentialivs two questions can be raised about a psychological 
unit's mission: (1) how well is it performing the array of tasks that 
constitute the mission and (2) what are the key factors’ that 
differentiate between success and failure (positive versus negative 
outcomes) ? The first question refers to evaluation; the second to 
internal research--i.e. designing and conducting research to identify 


the factors that most relate to mission success in contrast to those 


factors that have little or no impact. Such information can greatly 
increase the efficiency of a unit's operations. One administrator 
referred to this as ''Fine-tuning''. 


The immediate benefits of a systematic program of evaluation and 
internal research are deriving accountability information and 


improving efficiency of operations. However, another level of benefit 
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comes from engaging in such a systematic program. These activities 
require an intensive degree of thinking through one's tasks in order 
to develop the particular approach to evaluation and the design 
specifications for the internal research. The processes’ involved 


often lead to new perspectives on a unit's tasks and goals. 


Evaluation within psychological units - Each psychological unit 
has a set of specific tasks to perform. There are particular outcomes 
expected as a result of engaging in these tasks. The purpose of the 
evaluation is to determine how well the unit is performing each of its 
tasks in terms of the outcomes that are supposed to follow. 
Evaluation as presented here is a hard question and is not to be 


answered in terms of anecdotes or mythology. 


Evaluation must begin with establishing a set of operational 
definitions of the explicit outcomes that are expected as a result of 
each specific task. This requires collaboration with the unit's 
client. If a psychological unit is not looking down the road toa 
systematic evaluation, this stage might be left in a too generalized 
state, One can argue that such a specification of operational 
definitions of expected outcomes should be engaged in even if no 
systematic evaluation were anticipated. The probability is high, 
however, that the level of specification referred to here is not 


sufficient when the prospect of evaluation is not in the picture, 


In this Agency, access to information that would permit 


operational definition of specific outcomes sometimes cannot be 
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provided in the most direct form because of security considerations. 
In such cases a specific set of characteristics desired must be 
established and the evaluation becomes one of determining how 


effectively these qualities are identified. 


Outcomes do not necessarily lend themselves to categorization in 
terms of positive or negative. Some outcomes might have to be weighed 


in terms of a scaling approach, such as from excellent to poor. 


In any event, deriving operational definitions of outcomes 
desired or characteristics required can be a very difficult procedure. 
This process is necessary, we repeat, in describing the unit's program 
in the first place, not just for evaluation purposes. The prospect of 
an evaluation heightens the need for the procedure, One meaSure of 
the effectiveness of a psychological unit is the quality of its 
analysis in this particular area. Such analysis appears to be uneven 


among the psychological units in the Agency. 


Internal Research - Internal research within the respective 
psychological units would be directed toward determining the specific 
factors that are related to differentiating between positive and 


negative outcomes of their activities. (As stated, evaluation 


determines what the positive-negative outcome ratio is). 
Let uS assume that a job skills analysis approach has identified 
ten characteristics of an individual that must be drawn upon in order 
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to perform a given job. Procedures would have been developed that 
were designed to measure each of the ten characteristics. (In order 
to keep the terminology clear, ‘''validity'' refers to the degree each 


of the ten procedures actually measures the particular characteristic 
for which it was designed.) The question now becomes whether all ten 
of the characteristics, assuming each is measured validly, play equal 
roles in relating to the outcomes expected in the given job or 
operation. The most likely assumption is that the ten characteristics 
do not play such equal roles. There is most likely some sub-set of 
the ten characteristics that is most related to the desired outcome. 
The operational definitions of the desired outcomes would have been 


developed as a phase of evaluation. 


Information would have been collected, ii the evaluation phase, 
as to the ratio of positive or negative outcomes, or a_ scaling of 
degree of positive outcome (this, too, could be very difficult; 
especially where security considerations intervene). Statistical 
procedures such as multiple correlation, discriminant analysis and 
canonical correlation are available for producing the information 
sought--the identification of the critical sub-sets among the original 


ten characteristics. 


In discussing this one administrator immediately thought of an 
example. It had been assumed that one particular characteristic in a 


set of characteristics was critical in performing a certain job. 
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Further analysis demonstrated that the characteristic had little 
relation to the job but another one was highly related. The latter 
characteristic, luckily, had been included in the assessment 


procedures. 


Two important possibilities arise from this approach: (1) if a 
sub-set of the large array of characteristics is found to be critical, 
efficiency would come from concentrating upon the sub-set and (2) if 
none of the characteristics was related to desired outcomes, the 
particular program has been operating on false premises as to what 


constitute the necessary characteristics. 


Evaluation and internal research as the bases for improved 
communication - Previous reviews of the psychological units has led to 
the recommendation that there should be more communication among them. 
To date, efforts to accomplish this have not been too successful. 
Some say these attempts have been perfunctory; others say that''we 
don't have anything to talk about.'' One said that ''we don't have 


anything, or a project, to address.'' 


We have stressed that systematic evaluation and internal research 
are intrinsically difficult to develop. Security needs accentuate the 
difficulty of doing these things in the Agency, but it represents a 


task to which professionals in the psychological units could address 
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themselves. Some of the issues that readily come to mind and that 


could be the subject of such professional interchange are: 


25X11 


-~developing criteria for positive versus negative 
assessment of performance (or appropriate scaling 


systems). 


-developing procedures for obtaining data on perform- 


ance that do not risk violations of security. 


-how best to interact with managers in order to obtain 
from them their perspectives on performance require- 


ments. 


Even if the psychological units remain organizationally 
separated, as at present, each of them could contribute to such 


issues. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


The principal rationale behind this study was to determine 
whether or not the current structure within which psychologists are 
located and in which they operate is conducive to their effective 
utilization, or whether some alternative (or alternatives) would be 
better. It was clear from the assignment that one of the alternatives 
that has been given consideration in the past, and is of continuing 
interest to Agency management, is one of consolidating most, if not 


all, psychologists within a single organization. 


1. The work of psychologists in the Agency is sharply defined 
and follows closely the missions and functions of those organizations 
in which they are located. There is no significant overlap that would 
contribute to poor utilization on the basis of organizational 


location. 


Recommendation: Consolidation of psychologists within 
the Agency would serve no useful purpose, and “gould disrupt the 
effective conduct of present functions. There may be instances where 
some organizational adjustment would improve the utilization of 


psychologists. 
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2% Relative effectiveness in the utilization of psychologists 
was difficult to assess because there is no common Bpactics from one 
organization to another for collecting data that would support such an 
assessment. Information about effectiveness varies greatly and tends 
to be anecdotal and unsystematic. Psychologists appear to be busy and 
productively employed. Where caseload data is applicable, such as in 
the assessment function, the number of cases has increased by 25 to 50 
percent in the last few years as Agency recruitment efforts have 
intensified and as client organizations have requested assessments on 
more candidates. The study team believes that psychologists are not 
underutilized, but we are not in a position to judge whether, in the 
light of the Agency-wide or Directorate-wide priorities, they are at 


work on the most important tasks. 


Recommendation: Agency management should be able to 
determine if the distribution of effort in these psychological units 
coincides with Agency priorities. However, there is no particular 
reason to focus upon psychologists; this is a management 


responsibility applicable to all of the Agency's human resources. 


3. Systematic program evaluation is uneven among psychological 
units. As one might anticipate, the incentive for program evaluation 
is stronger among those units heavily involved in testing and 


assessment than in those concentrating more upon research, clinical 
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evaluation and counseling, and training. All units engage in some 
qualitative evaluation, such as user or client Satisfaction--i.e. does 
the client use the product, or does the client return for future work? 
In at least one instance, | | more systematic and methodologically 
relevant evaluation is hindered by security considerations. In this 
instance we believe that ways can be found to make data available for 
evaluation without revealing sensitive sources, but only if senior 
managers recognize the need and provide the organizational leverage to 


carry it out. 


Recommendation: Agency management Should require each 


psychological unit to develop and to conduct a systematic program of 


evaluation. 


4. Systematic internal research to determine those aspects of a 
unit's activities most related to success was notably lacking. Where 
there was program evaluation, it rarely was carried to the next step 
to systematically identify what elements in the particular task or 


objective contributed most to success. 


Recommendation: Agency management should encourage 
psychological units (and others) to undertake internal research. Such 
encouragement could be pursued through the budget, program review, and 


performance assessment processes. 
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5. Interchange among psychologists, across organizational lines, 
has been encouraged by Agency management from time to time, but has 
been sporadic. The process suffers from inadequate incentives and the 
press of immediate responsibilities or diverging interests. The study 
team Found a rich variety of activities that should be of interest 
among psychologists throughout the Agency. What is needed is a 
problem or issue to which all can make a contribution and which has 


elements of common interest. 


Recommendation: If Agency management makes a serious attempt 
to require or encourage systematic evaluation and internal research, 
psychologists, wherever located, will have abundant opportunity to 
seek mutual support and ideas in addressing challenges of methodology 


and technique to undertake these activities. 
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APPENDIX B 


Location of Professional 


Psychologists By Organization* 


*Occupational Classification is Noted Where it was Available, as Follows: 


PG=Psychologist-—General 
PC=Psychologist-Clinical 
PA=Psychologist—-Assessment 
25X1 PR=Psychologist~—Research 
BS=Behavioral Scientist 
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